Skip to content

[Feature] enable gz compression by default for StreamLoad#648

Merged
JNSimba merged 6 commits into
apache:masterfrom
JNSimba:enablegz
Apr 7, 2026
Merged

[Feature] enable gz compression by default for StreamLoad#648
JNSimba merged 6 commits into
apache:masterfrom
JNSimba:enablegz

Conversation

@JNSimba
Copy link
Copy Markdown
Member

@JNSimba JNSimba commented Apr 3, 2026

Summary

  • Enable gz compression by default for StreamLoad writes to reduce network transfer and improve write performance
  • Users can disable compression by explicitly setting 'sink.properties.compress_type' = ''
  • Covers all write paths: Flink SQL, DataStream API, CDC DatabaseSync, and Catalog
  • Added IT cases for default gz compression and explicit no-compression scenarios

Test plan

  • Verify default gz compression works (testTableDefaultGzFormat)
  • Verify explicit no-compression works (testTableNoCompressFormat)
  • Verify existing testTableGzFormat still passes
  • Verify all other sink IT cases pass with implicit default compression

🤖 Generated with Claude Code

JNSimba and others added 6 commits April 3, 2026 16:12
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Doris 2.1 does not support json compression, so explicitly set
compress_type to empty for json format test cases.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…fix checkstyle CI

Doris 2.1 does not support json compression, so disable default gz
compression for json format test cases. Also fix checkstyle CI to
install dependencies before running checkstyle:check.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@JNSimba JNSimba merged commit 06edd0c into apache:master Apr 7, 2026
8 of 9 checks passed
@JNSimba JNSimba mentioned this pull request Apr 7, 2026
JNSimba added a commit to apache/doris-website that referenced this pull request May 13, 2026
…compress_type (#3638)

## Versions

- [x] dev
- [x] 4.x
- [ ] 3.x
- [ ] 2.1 or older (not covered by version/language sync gate)

## Languages

- [x] Chinese
- [x] English
- [ ] Japanese candidate translation needed

## Docs Checklist

- [x] Checked by AI
- [ ] Test Cases Built
- [x] Updated required version and language counterparts, or explained
why not
- [x] If only one language changed, confirmed whether source/translation
counterparts need sync

## Summary

Since Flink Doris Connector 26.1.0, gz compression is enabled by default
for Stream Load (apache/doris-flink-connector#648). This PR documents
how to disable it:

- Append to the 26.1.0 release note: compression can be disabled by
setting `'sink.properties.compress_type' = ''`.
- Append the same note to the `sink.properties.*` description in the
connector configuration table so users can find it from the main config
docs.

Applied to all four locations: `docs/` (dev EN),
`versioned_docs/version-4.x/` (4.x EN), and the corresponding
`i18n/zh-CN` Chinese translations.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants